policies: treat SERIAL/LOCAL_SERIAL consistency as LWT for routing#887
policies: treat SERIAL/LOCAL_SERIAL consistency as LWT for routing#887mykaul wants to merge 2 commits into
Conversation
Statements with SERIAL or LOCAL_SERIAL consistency level are serialized through the Paxos path on the server, but TokenAwarePolicy only checked is_lwt() (from server prepare metadata) when deciding whether to skip replica shuffling. This meant serial-consistency reads could be routed with shuffled replicas instead of the deterministic order needed for optimal Paxos coordination. Now TokenAwarePolicy also checks the statement's consistency level and skips shuffling for SERIAL/LOCAL_SERIAL, matching LWT routing behavior. Fixes: scylladb#886
mykaul
force-pushed
the
fix-serial-consistency-lwt-routing
branch
from
76503d0 to
c14edfa
Compare
There was a problem hiding this comment.
Pull request overview
Updates TokenAwarePolicy routing behavior so statements executed with SERIAL / LOCAL_SERIAL consistency are treated like LWT-routing candidates (replica order preserved), matching expected Paxos/serial-path coordinator selection and fixing the replica-shuffle behavior described in #886.
Changes:
- Skip replica shuffling in
TokenAwarePolicywhenquery.consistency_levelisSERIALorLOCAL_SERIAL. - Add a unit test ensuring
shuffle()is not called for serial-consistency statements on both vnode and tablet-style metadata setups.
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.
| File | Description |
|---|---|
cassandra/policies.py |
Adjusts shuffle condition to preserve replica order for serial-consistency statements (in addition to LWT). |
tests/unit/test_policies.py |
Adds regression test asserting shuffle() is not invoked for SERIAL / LOCAL_SERIAL statements. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
sylwiaszunejko
left a comment
sylwiaszunejko
left a comment
There was a problem hiding this comment.
Looks good to me, @dkropachev could you also take a look?
sylwiaszunejko
left a comment
sylwiaszunejko
left a comment
There was a problem hiding this comment.
I missed that this part is not addressed:
when a statement is executed with SERIAL or LOCAL_SERIAL as regular
consistency (serial/Paxos read), retry policies must not downgrade it
to a non-serial consistency level like ONE or QUORUM, as that would
break the serial read guarantees.
Added. |
Add a guard in the retry execution path that prevents any retry policy from downgrading SERIAL/LOCAL_SERIAL to a non-serial consistency level, which would break serial read (Paxos) guarantees. Also add a unit test verifying DowngradingConsistencyRetryPolicy does not downgrade serial consistency on read timeout or unavailable.
mykaul
force-pushed
the
fix-serial-consistency-lwt-routing
branch
from
7347fe3 to
3b69e2a
Compare
dkropachev
left a comment
dkropachev
left a comment
There was a problem hiding this comment.
Great PR, but you forgot to adjust built-in retry policies to not even try to downgrade.
| if not ConsistencyLevel.is_serial(consistency_level): | ||
| original_cl = self.message.consistency_level | ||
| if ConsistencyLevel.is_serial(original_cl): | ||
| log.debug( | ||
| "Retry policy attempted to downgrade serial consistency %s to %s; " | ||
| "keeping original consistency level.", | ||
| ConsistencyLevel.value_to_name.get(original_cl, original_cl), | ||
| ConsistencyLevel.value_to_name.get(consistency_level, consistency_level)) | ||
| else: | ||
| self.message.consistency_level = consistency_level | ||
| else: | ||
| self.message.consistency_level = consistency_level |
There was a problem hiding this comment.
This is too over complicated, can you make it a single if-else please
There was a problem hiding this comment.
Will do. Actually AInhad a single if and I thought it be better to split.
Summary
TokenAwarePolicynow skips replica shuffling for statements withSERIALorLOCAL_SERIALconsistency level, treating them as LWT-routing candidatesis_lwt()(from server prepare metadata) was checked, meaning serial-consistency reads got shuffled replicas instead of deterministic Paxos-optimal orderingFixes: #886
Related: scylladb/java-driver#885